Search results for " peer review"
showing 10 items of 30 documents
Evaluation Framework for Analyzing the Applicability of Criteria Lists for the Selection of Requirements Management Tools Supporting Distributed Coll…
2016
Effective requirements management and enabling tools are critical for successfully developing and maintaining services and products. The identification and selection of an appropriate requirements management tool can be a costly, time-consuming, and error-prone undertaking especially in the context of software product line requirements management, requiring the tools to support both product and platform development activities that often involve geographically distributed, collaborating, and competing stakeholders. Criteria lists have been developed to facilitate the selection. This research (1) creates an evaluation framework to review the applicability of the lists for the selection of req…
Fragments of peer review: A quantitative analysis of the literature (1969-2015)
2018
This paper examines research on peer review between 1969 and 2015 by looking at records indexed from the Scopus database. Although it is often argued that peer review has been poorly investigated, we found that the number of publications in this field doubled from 2005. A half of this work was indexed as research articles, a third as editorial notes and literature reviews and the rest were book chapters or letters. We identified the most prolific and influential scholars, the most cited publications and the most important journals in the field. Co-authorship network analysis showed that research on peer review is fragmented, with the largest group of co-authors including only 2.1% of the wh…
Do editors have a silver bullet? an agent-based model of peer review
2014
This paper presents an agent-based model of peer review that looks at the effect of different editorial policies of referee selection. We tested four author/referee matching scenarios as follows: random selection of referees, selection of referees with a similar status to submission authors, selection of higher-skilled and lower skilled referees. We tested these scenarios against three types of referee behaviour, i.e., fair, unreliable and strategic and measured their implications for the quality and efficiency of the process. Results show that in case of randomness of referee judgment, any editorial policy is detrimental for peer review. If referees behave strategically, certain matching p…
When Competition Is Pushed Too Hard. An Agent-Based Model Of Strategic Behaviour Of Referees In Peer Review
2013
This paper examines the impact of strategic behaviour of referees on the quality and efficiency of peer review. We modelled peer review as a process based on knowledge asymmetry and subject to evaluation bias. We built two simulation scenarios to investigate largescale implications of referee behaviour and judgment bias. The first one was inspired by “the luck of the reviewer draw” idea. In this case, we assumed that referees randomly fell into Type I and Type II errors, i.e., recommending submissions of low quality to be published or recommending against the publishing of submissions which should have been published. In the second scenario, we assumed that certain referees tried intentiona…
A simulation of disagreement for control of rational cheating in peer review
2013
Understanding the peer review process could help research and shed light on the mechanisms that underlie crowdsourcing. In this paper, we present an agent-based model of peer review built on three entities - the paper, the scientist and the conference. The system is implemented on a BDI platform (Jason) that allows to define a rich model of scoring, evaluating and selecting papers for conferences. Then, we propose a programme committee update mechanism based on disagreement control that is able to remove reviewers applying a strategy aimed to prevent papers better than their own to be accepted (rational cheating). We analyze a homogeneous scenario, where all conferences aim to the same leve…
An Information Systems Design Product Theory for Software Project Estimation and Measurement Systems
2009
There is relatively little research on software Project Estimation and Measurement Systems (PEMS). Commercial PEMS vary in functionality and effective- ness. Their intended users thus do not know what to ex- pect from PEMS and how to evaluate them. This paper creates an information system design product theory for the class of PEMS that prescribes the meta-requirements, the meta-design, and applicable theories for all products within the class. Meta-requirements and the meta-design are derived from the project estimation and measurement literature, experiences obtained during more than ten years of empirical work in Finnish Software Measurement Association, and a commercially available PEMS.
Look for methods, not conclusions
2019
Qualunque sia l'esito effettivo di un'indagine su un caso di presunta manipolazione dei dati, ci sono alcuni principi fermi che sono sempre veri e alcuni errori che devono essere evitati.
Effects of seniority, gender and geography on the bibliometric output and collaboration networks of European Research Council (ERC) grant recipients.
2019
Assessing the success and performance of researchers is a difficult task, as their grant output is influenced by a series of factors, including seniority, gender and geographical location of their host institution. In order to assess the effects of these factors, we analysed the publication and citation outputs, using Scopus and Web of Science, and the collaboration networks of European Research Council (ERC) starting (junior) and advanced (senior) grantees. For this study, we used a cohort of 355 grantees from the Life Sciences domain of years 2007-09. While senior grantees had overall greater publication output, junior grantees had a significantly greater pre-post grant award increase in …
Assessing the format and content of journal published and non-journal published rapid review reports: A comparative study
2020
Background As production of rapid reviews (RRs) increases in healthcare, knowing how to efficiently convey RR evidence to various end-users is important given they are often intended to directly inform decision-making. Little is known about how often RRs are produced in the published or unpublished domains, and what and how information is structured. Objectives To compare and contrast report format and content features of journal-published (JP) and non-journal published (NJP) RRs. Methods JP RRs were identified from key databases, and NJP RRs were identified from a grey literature search of 148 RR producing organizations and were sampled proportionate to cluster size by organization and pro…
Software Business in the Telecommunications Sector
2012
Operations and Business Support Systems (OSS/BSS) software of Communication Service Providers (CSP's) can be developed internally within the CSP or acquired from a Software Vendor. The software industry lifecycle model hypothesizes that software development is internal in the beginning of the industry's lifecycle, and that the share of external products increases when it matures. Empirical evidence shows signs of the OSS/BSS software industry approaching maturity. Current and future developments of the industry include the possibilities of utilizing the Software-as-a-Service (SaaS) model and Open Source software (OSSw). Both have gained increasing interest by the CSP's. However, the relativ…